Identifying and managing problematic trials: a Research Integrity Assessment (RIA) tool for randomized controlled trials in evidence synthesis.

Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms to identify and manage problematic studies to avoid misleading findings. As evidence synthesis authors of the Cochrane COVID-19 review on ivermectin, we identified many problematic studies in terms of research integrity and regulatory compliance. Through iterative discussion, we developed a Research Integrity Assessment (RIA) tool for randomized controlled trials (RCTs) for the update of this Cochrane review. In this paper, we explain the rationale and application of the RIA tool in this case study. RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results. RIA was used in the Cochrane review as part of the eligibility check during screening of potentially eligible studies. Problematic studies were excluded and studies with open questions were held in awaiting classification until clarified. RIA decisions were made independently by two authors and reported transparently. Using the RIA tool resulted in the exclusion of >40% of studies in the first update of the review. RIA is a complementary tool prior to assessing 'Risk of Bias' aiming to establish the integrity and authenticity of studies. RIA provides a platform for urgent development of a standard approach to identifying and managing problematic studies. This article is protected by copyright. All rights reserved.

Nirmatrelvir combined with ritonavir for preventing and treating COVID‐19

Objectives This is a protocol for a Cochrane Review (intervention). The objectives are as follows: To assess the efficacy and safety of nirmatrelvir/ritonavir (Paxlovid®) plus standard of care compared to standard of care with or without placebo, or any other proven intervention for treating COVID‐19 and for preventing SARS‐CoV‐2 infection.

Nirmatrelvir combined with ritonavir for preventing and treating COVID-19.

BACKGROUND: Oral nirmatrelvir/ritonavir (Paxlovid®) aims to avoid severe COVID-19 in asymptomatic people or those with mild symptoms, thereby decreasing hospitalization and death. Due to its novelty, there are currently few published study results. It remains to be evaluated for which indications and patient populations the drug is suitable.  OBJECTIVES: To assess the efficacy and safety of nirmatrelvir/ritonavir (Paxlovid®) plus standard of care compared to standard of care with or without placebo, or any other intervention for treating COVID-19 and for preventing SARS-CoV-2 infection. To explore equity aspects in subgroup analyses. To keep up to date with the evolving evidence base using a living systematic review (LSR) approach and make new relevant studies available to readers in-between publication of review updates. SEARCH METHODS: We searched the Cochrane COVID-19 Study Register, Scopus, and WHO COVID-19 Global literature on coronavirus disease database, identifying completed and ongoing studies without language restrictions and incorporating studies up to 11 July 2022.  This is a LSR. We conduct monthly update searches that are being made publicly available on the open science framework (OSF) platform. SELECTION CRITERIA: Studies were eligible if they were randomized controlled trials (RCTs) comparing nirmatrelvir/ritonavir plus standard of care with standard of care with or without placebo, or any other intervention for treatment of people with confirmed COVID-19 diagnosis, irrespective of disease severity or treatment setting, and for prevention of SARS-CoV-2 infection. We screened all studies for research integrity. Studies were ineligible if they had been retracted, or if they were not prospectively registered including appropriate ethics approval. DATA COLLECTION AND ANALYSIS: We followed standard Cochrane methodology and used the Cochrane risk of bias 2 tool. We rated the certainty of evidence using the GRADE approach for the following outcomes: 1. to treat outpatients with mild COVID-19; 2. to treat inpatients with moderate-to-severe COVID-19: mortality, clinical worsening or improvement, quality of life, (serious) adverse events, and viral clearance; 3. to prevent SARS-CoV-2 infection in post-exposure prophylaxis (PEP); and 4. pre-exposure prophylaxis (PrEP) scenarios: SARS-CoV-2 infection, development of COVID-19 symptoms, mortality, admission to hospital, quality of life, and (serious) adverse events. We explored inequity by subgroup analysis for elderly people, socially-disadvantaged people with comorbidities, populations from LICs and LMICs, and people from different ethnic and racial backgrounds. MAIN RESULTS: As of 11 July 2022, we included one RCT with 2246 participants in outpatient settings with mild symptomatic COVID-19 comparing nirmatrelvir/ritonavir plus standard of care with standard of care plus placebo. Trial participants were unvaccinated, without previous confirmed SARS-CoV-2 infection, had a symptom onset of no more than five days before randomization, and were at high risk for progression to severe disease. Prohibited prior or concomitant therapies included medications highly dependent on CYP3A4 for clearance and CYP3A4 inducers.  We identified eight ongoing studies. Nirmatrelvir/ritonavir for treating COVID-19 in outpatient settings with asymptomatic or mild disease For the specific population of unvaccinated, high-risk patients nirmatrelvir/ritonavir plus standard of care compared to standard of care plus placebo may reduce all-cause mortality at 28 days (risk ratio (RR) 0.04, 95% confidence interval (CI) 0.00 to 0.68; 1 study, 2224 participants; estimated absolute effect: 11 deaths per 1000 people receiving placebo compared to 0 deaths per 1000 people receiving nirmatrelvir/ritonavir; low-certainty evidence, and admission to hospital or death within 28 days (RR 0.13, 95% CI 0.07 to 0.27; 1 study, 2224 participants; estimated absolute effect: 61 admissions or deaths per 1000 people receiving placebo compared to eight admissions or deaths per 1000 people receiving nirmatrelvir/ritonavir; low-certainty evidence). Nirmatrelvir/ritonavir plus standard of care may reduce serious adverse events during the study period compared to standard of care plus placebo (RR 0.24, 95% CI 0.15 to 0.41; 1 study, 2224 participants; low-certainty evidence). Nirmatrelvir/ritonavir plus standard of care probably has little or no effect on treatment-emergent adverse events (RR 0.95, 95% CI 0.82 to 1.10; 1 study, 2224 participants; moderate-certainty evidence), and probably increases treatment-related adverse events such as dysgeusia and diarrhoea during the study period compared to standard of care plus placebo (RR 2.06, 95% CI 1.44 to 2.95; 1 study, 2224 participants; moderate-certainty evidence). Nirmatrelvir/ritonavir plus standard of care probably decreases discontinuation of study drug due to adverse events compared to standard of care plus placebo (RR 0.49, 95% CI 0.30 to 0.80; 1 study, 2224 participants; moderate-certainty evidence). No study results were identified for improvement of clinical status, quality of life, and viral clearance.  Subgroup analyses for equity Most study participants were younger than 65 years (87.1% of the : modified intention to treat (mITT1) population with 2085 participants), of white ethnicity (71.5%), and were from UMICs or HICs (92.1% of study centres). Data on comorbidities were insufficient.  The outcome 'admission to hospital or death' was investigated for equity: age (< 65 years versus ≥ 65 years) and ethnicity (Asian versus Black versus White versus others). There was no difference between subgroups of age. The effects favoured treatment with nirmatrelvir/ritonavir for the White ethnic group. Estimated effects in the other ethnic groups included the line of no effect (RR = 1). No subgroups were reported for comorbidity status and World Bank country classification by income level. No subgroups were reported for other outcomes. Nirmatrelvir/ritonavir for treating COVID-19 in inpatient settings with moderate to severe disease No studies available. Nirmatrelvir/ritonavir for preventing SARS-CoV-2 infection (PrEP and PEP) No studies available. AUTHORS' CONCLUSIONS: There is low-certainty evidence that nirmatrelvir/ritonavir reduces the risk of all-cause mortality and hospital admission or death based on one trial investigating unvaccinated COVID-19 participants without previous infection that were at high risk and with symptom onset of no more than five days. There is low- to moderate-certainty evidence that nirmatrelvir/ritonavir is safe in people without prior or concomitant therapies including medications highly dependent on CYP3A4. Regarding equity aspects, except for ethnicity, no differences in effect size and direction were identified. No evidence is available on nirmatrelvir/ritonavir to treat hospitalized people with COVID-19 and to prevent a SARS-CoV-2 infection. We will continually update our search and make search results available on OSF.

Pragmatic and open science-based solution to a current problem in the reporting of living systematic reviews.

Living systematic reviews (LSRs) are an increasingly common approach to keeping reviews up to date, in which new relevant studies are incorporated as they become available, so as to inform healthcare policy and practice in a timely manner. While journal publishers have been exploring the publication of LSRs using different updating and publishing approaches, readers cannot currently assess if the evidence underpinning a published LSR is up to date, as neither the search details, the selection process, nor the list of identified studies is made available between the publication of updates. We describe a new method to transparently report the living evidence surveillance process that occurs between published LSR versions. We use the example of the living Cochrane Review on nirmatrelvir combined with ritonavir (Paxlovid) for preventing and treating COVID-19 to illustrate how this can work in practice. We created a publicly accessible spreadsheet on the Open Science Framework platform, linking to the living Cochrane Review, that details the search and study selection process, enabling readers to track the progress of eligible ongoing or completed studies. Further automation of the evidence surveillance process should be explored.

Anticoagulation in COVID-19 patients - An updated systematic review and meta-analysis.

BACKGROUND: Thromboembolic events are common complications of COVID-19. Clinical study results on safety and efficacy of anticoagulation in COVID-19 are controversial. MATERIAL AND METHODS: This report updates our systematic review and random-effects meta-analysis on randomized controlled trials (RCTs) comparing standard prophylactic anticoagulation and intermediate or therapeutic anticoagulation in COVID-19 patients. We searched eligible studies for the update up to 4 February 2022 by weekly monitoring of RCTs in the Cochrane COVID-19 Study Register. Certainty of evidence was assessed using GRADE (Grading of Recommendations Assessment, Development and Evaluation). RESULTS: For this update we included five new trials; a total of 13 RCTs with 7364 patients. Certainty of evidence was very low to low. We are uncertain whether low-dose prophylactic anticoagulation is favoured over placebo or no anticoagulation in the outpatient- or post-discharge-setting. In hospitalized patients with moderate and severe COVID-19, intermediate-dose anticoagulation may have little or no effect on thrombotic events or death (RR 1.03, 95 % CI 0.86-1.24), but may increase severe bleeding non-significantly (RR 1.48, 95 % CI 0.53-4.15). Therapeutic-dose anticoagulation may decrease thrombotic events or deaths in hospitalized patients with moderate COVID-19 (RR 0.64, 95 % CI 0.38-1.07; fixed-effect model RR 0.72, 95 % CI 0.57-0.91), but may have little or no effect in patients with severe disease (RR 0.98, 95 % CI 0.86-1.12). With therapeutic-dose anticoagulation, the risk of major bleeding may increase regardless of COVID-19 severity (RR 1.78, 95 % CI 1.15-2.74). CONCLUSIONS: Hospitalized, moderately ill COVID-19 patients may benefit from therapeutic-dose anticoagulation, while critically ill patients may not. Risk of major bleeding must be considered.

Ivermectin for preventing and treating COVID-19.

BACKGROUND: Ivermectin, an antiparasitic agent, inhibits the replication of viruses in vitro. The molecular hypothesis of ivermectin's antiviral mode of action suggests an inhibitory effect on severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) replication in early stages of infection. Currently, evidence on ivermectin for prevention of SARS-CoV-2 infection and COVID-19 treatment is conflicting. OBJECTIVES: To assess the efficacy and safety of ivermectin plus standard of care compared to standard of care plus/minus placebo, or any other proven intervention for people with COVID-19 receiving treatment as inpatients or outpatients, and for prevention of an infection with SARS-CoV-2 (postexposure prophylaxis). SEARCH METHODS: We searched the Cochrane COVID-19 Study Register, Web of Science (Emerging Citation Index and Science Citation Index), WHO COVID-19 Global literature on coronavirus disease, and HTA database weekly to identify completed and ongoing trials without language restrictions to 16 December 2021. Additionally, we included trials with > 1000 participants up to April 2022. SELECTION CRITERIA: We included randomized controlled trials (RCTs) comparing ivermectin to standard of care, placebo, or another proven intervention for treatment of people with confirmed COVID-19 diagnosis, irrespective of disease severity or treatment setting, and for prevention of SARS-CoV-2 infection. Co-interventions had to be the same in both study arms.  For this review update, we reappraised eligible trials for research integrity: only RCTs prospectively registered in a trial registry according to WHO guidelines for clinical trial registration were eligible for inclusion. DATA COLLECTION AND ANALYSIS: We assessed RCTs for bias, using the Cochrane RoB 2 tool. We used GRADE to rate the certainty of evidence for outcomes in the following settings and populations: 1) to treat inpatients with moderate-to-severe COVID-19, 2) to treat outpatients with mild COVID-19 (outcomes: mortality, clinical worsening or improvement, (serious) adverse events, quality of life, and viral clearance), and 3) to prevent SARS-CoV-2 infection (outcomes: SARS-CoV-2 infection, development of COVID-19 symptoms, admission to hospital, mortality, adverse events and quality of life). MAIN RESULTS: We excluded seven of the 14 trials included in the previous review version; six were not prospectively registered and one was non-randomized. This updated review includes 11 trials with 3409 participants investigating ivermectin plus standard of care compared to standard of care plus/minus placebo. No trial investigated ivermectin for prevention of infection or compared ivermectin to an intervention with proven efficacy. Five trials treated participants with moderate COVID-19 (inpatient settings); six treated mild COVID-19 (outpatient settings). Eight trials were double-blind and placebo-controlled, and three were open-label. We assessed around 50% of the trial results as low risk of bias. We identified 31 ongoing trials. In addition, there are 28 potentially eligible trials without publication of results, or with disparities in the reporting of the methods and results, held in 'awaiting classification' until the trial authors clarify questions upon request. Ivermectin for treating COVID-19 in inpatient settings with moderate-to-severe disease We are uncertain whether ivermectin plus standard of care compared to standard of care plus/minus placebo reduces or increases all-cause mortality at 28 days (risk ratio (RR) 0.60, 95% confidence interval (CI) 0.14 to 2.51; 3 trials, 230 participants; very low-certainty evidence); or clinical worsening, assessed by participants with new need for invasive mechanical ventilation or death at day 28 (RR 0.82, 95% CI 0.33 to 2.04; 2 trials, 118 participants; very low-certainty evidence); or serious adverse events during the trial period (RR 1.55, 95% CI 0.07 to 35.89; 2 trials, 197 participants; very low-certainty evidence). Ivermectin plus standard of care compared to standard of care plus placebo may have little or no effect on clinical improvement, assessed by the number of participants discharged alive at day 28 (RR 1.03, 95% CI 0.78 to 1.35; 1 trial, 73 participants; low-certainty evidence); on any adverse events during the trial period (RR 1.04, 95% CI 0.61 to 1.79; 3 trials, 228 participants; low-certainty evidence); and on viral clearance at 7 days (RR 1.12, 95% CI 0.80 to 1.58; 3 trials, 231 participants; low-certainty evidence). No trial investigated quality of life at any time point. Ivermectin for treating COVID-19 in outpatient settings with asymptomatic or mild disease Ivermectin plus standard of care compared to standard of care plus/minus placebo probably has little or no effect on all-cause mortality at day 28 (RR 0.77, 95% CI 0.47 to 1.25; 6 trials, 2860 participants; moderate-certainty evidence) and little or no effect on quality of life, measured with the PROMIS Global-10 scale (physical component mean difference (MD) 0.00, 95% CI -0.98 to 0.98; and mental component MD 0.00, 95% CI -1.08 to 1.08; 1358 participants; high-certainty evidence). Ivermectin may have little or no effect on clinical worsening, assessed by admission to hospital or death within 28 days (RR 1.09, 95% CI 0.20 to 6.02; 2 trials, 590 participants; low-certainty evidence); on clinical improvement, assessed by the number of participants with all initial symptoms resolved up to 14 days (RR 0.90, 95% CI 0.60 to 1.36; 2 trials, 478 participants; low-certainty evidence); on serious adverse events (RR 2.27, 95% CI 0.62 to 8.31; 5 trials, 1502 participants; low-certainty evidence); on any adverse events during the trial period (RR 1.24, 95% CI 0.87 to 1.76; 5 trials, 1502 participants; low-certainty evidence); and on viral clearance at day 7 compared to placebo (RR 1.01, 95% CI 0.69 to 1.48; 2 trials, 331 participants; low-certainty evidence). None of the trials reporting duration of symptoms were eligible for meta-analysis. AUTHORS' CONCLUSIONS: For outpatients, there is currently low- to high-certainty evidence that ivermectin has no beneficial effect for people with COVID-19. Based on the very low-certainty evidence for inpatients, we are still uncertain whether ivermectin prevents death or clinical worsening or increases serious adverse events, while there is low-certainty evidence that it has no beneficial effect regarding clinical improvement, viral clearance and adverse events. No evidence is available on ivermectin to prevent SARS-CoV-2 infection. In this update, certainty of evidence increased through higher quality trials including more participants. According to this review's living approach, we will continually update our search.

Safety and Efficacy of Intermediate- and Therapeutic-Dose Anticoagulation for Hospitalised Patients with COVID-19: A Systematic Review and Meta-Analysis.

BACKGROUND: COVID-19 patients are at high thrombotic risk. The safety and efficacy of different anticoagulation regimens in COVID-19 patients remain unclear. METHODS: We searched for randomised controlled trials (RCTs) comparing intermediate- or therapeutic-dose anticoagulation to standard thromboprophylaxis in hospitalised patients with COVID-19 irrespective of disease severity. To assess efficacy and safety, we meta-analysed data for all-cause mortality, clinical status, thrombotic event or death, and major bleedings. RESULTS: Eight RCTs, including 5580 patients, were identified, with two comparing intermediate- and six therapeutic-dose anticoagulation to standard thromboprophylaxis. Intermediate-dose anticoagulation may have little or no effect on any thrombotic event or death (RR 1.03, 95% CI 0.86-1.24), but may increase major bleedings (RR 1.48, 95% CI 0.53-4.15) in moderate to severe COVID-19 patients. Therapeutic-dose anticoagulation may decrease any thrombotic event or death in patients with moderate COVID-19 (RR 0.64, 95% CI 0.38-1.07), but may have little or no effect in patients with severe disease (RR 0.98, 95% CI 0.86-1.12). The risk of major bleedings may increase independent of disease severity (RR 1.78, 95% CI 1.15-2.74). CONCLUSIONS: Certainty of evidence is still low. Moderately affected COVID-19 patients may benefit from therapeutic-dose anticoagulation, but the risk for bleeding is increased.